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Applicants hereby petition the Director to set aside the Examiner's Election Requirement 
dated October 17, 2007, because: (I) it would not be a serious burden for the Examiner to 
continue examining the application on the merits without the Election Requirement; (II) the 
Examiner is not authorized under the Rules to issue the Election Requirement; (III) the Examiner 
has failed to establish a prima facie case for requiring an election of the claims; and (IV) no valid 
reason exists for dividing among the asserted "species". 

The pending claims are reproduced in the attached Claims Appendix. 



I. IT WOULD NOT BE A SERIOUS BURDEN FOR THE EXAMINER TO CONTINUE 
EXAMINING THE APPLICATION ON THE MERITS 

MPEP § 803.01 provides that (emphasis added): 



PETITION UNDER 37 CFR 1.181 



If the search and examination of an entire application can be made 
without serious burden, the examiner must examine it on the 
merits, even though it includes claims to independent or distinct 
inventions. 
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I hereby certify that this document is being transmitted to the Patent and 
Trademark Office via electronic filing. 



December 17, 2007 
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(Signature of person mailing papers) 



Edouard Garcia 



(Typed or printed name of person mailing papers) 
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The Examiner already has searched and examined the entire application on the merits 
before issuing the above-mentioned Election Requirement. The Examiner's Election 
Requirement therefore is improper at the present stage of prosecution because it would not be a 
serious burden for the Examiner to continue examining the application on the merits without the 
Election Requirement. 

A summary of the prosecution history of the application is set forth below. 

1/23/04 The application was filed with claims 1-64, where claims 1, 31, 32, 
33, 63, and 64 were independent claims, claims 2-30 depended 
from claim 1 and claims 43-62 depended from claim 33. 

3/22/07 The Examiner mailed the first Office action on the merits with the 
following claim rejections: 

claims rejected 32 and 64 under 35 U.S.C. § 101 ; 

claims rejected 1, 2, 4-12, 15, 19, 20, and 22-31 under 35 
U.S.C. § 102(e) over Pryor (U.S. 7,042,440); 

claims rejected 33, 34, 37, 39, 40, and 63 rejected under 35 
U.S.C. § 102(e) over Pulli (U.S. 6,771,294); 

claims 33 and 35 rejected under 35 U.S.C. § 102(a) over 
Kurtenbach (U.S. 2003/0142067); 

claim 13 rejected under 35 U.S.C. § 103(a) over Pyror; 

claim 3 rejected under 35 U.S.C. § 103(a) over Kumra (u.S. 
6,204,852); 

claims 16-18 and 21 rejected under 35 U.S.C. § 103(a) over 
Pyror in view of Schmalstieg (U.S. 6,842,175) 

claims 36 and 38 rejected under 35 U.S.C. § 103(a) over 
Pulli; 

claims 41-48, 52-60, and 62 rejected under 35 U.S.C. § 
103(a) over Pulli in view of Pyror; and 

claims 49-51 and 54 rejected under 35 U.S.C. § 103(a) over 
Pulli in view of Pryor and Scmmalstieg. 

6/21/07 Applicants mailed an Amendment in which: 
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independent claim 1 was amended to include the elements 
of claim 7, among other elements; 

claim 7 was canceled; 

claims 2-6, 9, 1 1, 13, 15-27, and 29 were left in their 
original form; 

claims 8, 10, 12, 14, and 28 were amended; 

claim 30 was rewritten in independent form; 

independent claims 31 and 32 were amended to essentially 
track claim 1; 

independent claim 33 was amended to incorporate the 
elements of claim 38; 

claim 38 was canceled; 

claims 34-37 and 39-63 were left in their original form; 

independent claims 63 and 64 were amended to essentially 
track claim 33; and 

dependent claim 65 was added. 

10/17/07 The Examiner mailed the election requirement that is the subject of 
the instant Petition. 

The Examiner already has fully and completely searched and examined the subject matter 
of all of the pending claims on the merits in the first Office action dated March 22, 2007. The 
scope of the subject matter recited in the pending claims is essentially the same as the subject 
matter scope of the claims that already have been fully and completely examined on the merits. 
Indeed, independent claims 33, 63, and 64 have been amended to incorporate only the elements 
of canceled claim 38, which the Examiner examined in the first Office action. Independent 
claims 1,31, and 32 have been amended to incorporate the elements of canceled claim 7, which 
the Examiner examined in the first Office action. Although independent claims 1,31, and 32 
also have been amended to include elements that refer to "capture times" instead of "reference 
times 55 as originally recited, these amendments did not change the invention defined in these 
claims to a species that is different from the species originally defined by these claims. 
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Therefore, it would not be a serious burden for the Examiner to continue examining the 
application on the merits, regardless of whether the application includes claims that are 
independent and distinct. 

For at least this reason, Applicants request that the Director set aside the Election 
Requirement dated October 17, 2007. 

II. THE EXAMINER IS NOT AUTHORIZED TO ISSUE THE ELECTION 
REQUIREMENT 

The Examiner is not authorized to issue the election requirement data October 17, 2007, 
because: (A) the Rules do not permit the Examiner to issue an election requirement where there 
is no generic claim; and (B) the Rules do not permit the Examiner to issue an election 
requirement that is incomplete. 

(A) The Rules Do Not Permit The Examiner To Issue An Election Requirement 
Where There Is No Generic Claim 

The election requirement is traversed because the Examiner is not authorized to require 
the proposed election of "species". In particular, 37 CFR 1.146, which authorizes the Examiner 
to require an election of species, applies only to "an application containing a generic claim to a 
generic invention (genus) and claims to more than one patentably distinct species embraced 
thereby." The instant application, however, currently does not contain a generic claim that 
embraces all the "species" I and II identified by the Examiner. Without such a generic claim, the 
"species" I and II do not constitute species to which the claims properly can be restricted under 
37 CFR 1.146 (see MPEP § 806.04). 

For at least this additional reason, Applicants request that the Director set aside the 
Election Requirement dated October 17, 2007. 
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(B) The Rules Do Not Permit The Examiner To Issue An Incomplete Election 
Requirement 



In the Office action dated October 17, 2007, the Examiner has required applicant to elect 
between the "species" I and "species" II. In accordance with the Examiner's position, claims 1- 
32 correspond to "species" I, and claims 33-37 and 64 correspond to "species" II. The 
Examiner, however, has not provided any indication of how the pending claims 38-63 and 65 
would be treated under the election requirement. Therefore, the Examiner has failed to provide a 
clear and detailed record of the election requirement as mandated under MPEP §§814 and 815. 

For at least this additional reason, Applicants request that the Director set aside the 
Election Requirement dated October 17, 2007. 



III. THE EXAMINER HAS FAILED TO ESTABLISH A PRIMA FACIE CASE FOR 
REQUIRING THE PROPOSED ELECTION OF CLAIMS 



In the Office action dated May 22, 2007, the Examiner has required an election between 
Species I (claims 1-32) and Species II (claims 33-37 and 64). 

The specification discloses that the asserted "species" I and II are related. For example, 
the specification discloses an embodiment that includes the elements of the asserted "species" I 
(i.e., a spatiotemporal input data structure as defined in independent claims 1,31, and 32) and the 
elements of the asserted "species" II (i.e., an arrangement of display space, viewing space, and 
interactive space as defined in independent claims 33, 63, and 64) (see, e.g.: page 24, lines 8-19; 
page 4, line 29 - page 5, line 9; page 5, lines 1 1-13 and page 6, lines 3-12; and FIGS. 1 and 14). 

In the case of related inventions, MPEP § 806.05Q) explains that (emphasis added): 

To support a requirement for restriction between two or more 
related product inventions, or between two or more related process 
inventions, both two-way distinctness and reasons for insisting on 
restriction are necessary, i.e., separate classification, status in the 
art, or field of search. See MPEP § 808.02. See MPEP § 806.05(c) 
for an explanation of the requirements to establish two-way 
distinctness as it applies to inventions in a 
combination/subcombination relationship. For other related 
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product inventions, or related process inventions, the inventions 
are distinct if 

(A) the inventions as claimed do not overlap in scope, i.e., are 
mutually exclusive; 

(B) the inventions as claimed are not obvious variants; and 

(C) the inventions as claimed are either not capable of use together 
or can have a materially different design, mode of operation, 
function, or effect. See MPEP § 802.01. 

The burden is on the examiner to provide an example to support 
the determination that the inventions are distinct, but the example 
need not be documented. 

The only explanation given by the Examiner in support of the election requirement is 

that: 

The species are independent or distinct because Species II has no 
requirement of needing a spatiotemporal data structure as 
described in the Species I, and Species I has no specific 
requirements of the arrangement of display space, viewing space, 
and interactive space, as described in Species II. 

Thus, the Examiner's election requirement is premised solely on his conclusion that the 
asserted "species" are mutually exclusive from each other. Under MPEP § 806.050), however, 
mutual exclusivity is insufficient by itself to establish that the species are either independent or 
distinct. Therefore, the Examiner has not made the showing required under MPEP § 806.05Q 
and, consequently, has not established a prima facie basis for requiring election between each of 
the asserted "species" I and II. 

For at least this additional reason, Applicants request that the Director set aside the 
Election Requirement dated October 17, 2007. 

IV. No Valid Reason Exists for Dividing Among the Related Inventions 

In general, if "the classification is the same and the field of search is the same and there is 
no clear indication of separate future classification and field of search, no reasons exist for 
dividing among related inventions" (MPEP § 808.02). 
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The Examiner has not provided any basis for believing that the classification of the 
asserted "species" I and II is not the same, nor has the Examiner provided any basis for believing 
that the field of search of the asserted "species" I and II is not the same. To the contrary, the fact 
that the Examiner already has examined the subject matter corresponding to the asserted 
"species" I and II in the first Office action (see § I above) evidences the fact that the 
classification of the asserted "species" I and II is the same and the field of search of the asserted 
"species" I and II is the same. The Examiner also has not provided any clear indication that the 
classification of the asserted "species" I and II would be different in the future. Thus, the 
Examiner has not shown that separate examinations are required for the asserted "species" I and 
II. Accordingly, under MPEP § 808.02 "no reasons exist for dividing among related inventions" 
and the election requirement should be withdrawn. 

For at least this additional reason, Applicants request that the Director set aside the 
Election Requirement dated October 17, 2007. 



V. CONCLUSION 



For at least the reasons explained above, Applicants request that the Director set aside the 
Examiner's Election Requirement dated October 17, 2007. 

Charge any excess fees or apply any credits to Deposit Account No. 08-2025. 

Respectfully submitted, 



Date: December 17. 2007 




Edouard Garcia 
Reg. No. 38,461 

Telephone No.: (650) 631-6591 



Please direct all correspondence to: 

Hewlett-Packard Company 

Intellectual Property Administration 

Legal Department, M/S 35 

P.O. Box 272400 

Fort Collins, CO 80528-9599 
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CLAIMS APPENDIX 

The claims pending in Application Serial No. 10/764,423 are reproduced in the following 
Listing of Claims: 

Listing of Claims : 

Claim 1 (previously presented): A method of interfacing with a machine, comprising: 
at each of multiple capture times, contemporaneously acquiring a respective image from 

each of multiple fields of view defining an interactive space to create a respective set of 

contemporaneous images; 

detecting an input target in the acquired images; 

computing two-dimensional coordinates of the input target detected in the acquired 
images; 

constructing a spatiotemporal input data structure linking each of the capture times to the 
computed two-dimensional coordinates of the input target in each of the contemporaneously 
acquired images in a respective one of the sets of contemporaneous images; 

processing the spatiotemporal input data structure to identify an input instruction; and 

executing the identified input instruction on the machine. 

Claim 2 (original): The method of claim 1, wherein images of the interactive space are 
acquired from at least one stereoscopic pair of fields of view directed along substantially parallel 
axes intersecting the interactive space. 

Claim 3 (original): The method of claim 1, wherein images of the interactive space are 
acquired from at least three different fields of view. 

Claim 4 (original): The method of claim 1, wherein detecting the input target in the 
acquired images comprises comparing values of pixels in the acquired images to at least one 
threshold pixel value. 
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Claim 5 (original): The method of claim 4, wherein computing coordinates of the input 
target comprises computing coordinates of centroids of respective groups of pixels in the 
acquired images with values greater than the at least one threshold pixel value. 

Claim 6 (original): The method of claim 4, wherein detecting the input target in the 
acquired images comprises segmenting foreground pixels from background pixels in the 
acquired images. 

Claim 7 (canceled) 

Claim 8 (previously presented): The method of claim 1, further comprising computing 
calibration parameters for the multiple fields of view. 

Claim 9 (original): The method of claim 8, wherein computing coordinates of the 
detected input target comprises computing three-dimensional coordinates of the input target in 
the interactive space based on the computed two-dimensional coordinates and the computed 
calibration parameters. 

Claim 10 (previously presented): The method of claim 9, wherein the spatiotemporal 
input data structure additionally links each of the capture times to respective three-dimensional 
coordinates of the input target computed from the computed two-dimensional coordinates of the 
input target in each of the contemporaneously acquired images in a respective one of the sets of 
contemporaneous images. 

Claim 1 1 (original): The method of claim 1, further comprising acquiring color values of 
the detected input target in the acquired images. 

Claim 12 (previously presented): The method of claim 11, wherein the spatiotemporal 
input data structure additionally links each of the capture times to respective color values 
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determined from the contemporaneously acquired images in a respective one of the sets of 
contemporaneous images. 

Claim 13 (original): The method of claim 1, wherein the spatiotemporal input data 
structure is constructed in the form of a linked list of data records. 

Claim 14 (previously presented): The method of claim 1, wherein processing the 
spatiotemporal input data structure comprises identifying traces of the input target in the 
interactive space, each trace being defined by a respective set of connected data items in the 
spatiotemporal input data structure. 

Claim 15 (original): The method of claim 14, wherein identifying traces comprises 
detecting state change events and segmenting traces based on detected state change events. 

Claim 16 (original): The method of claim 14, wherein identifying traces comprises 
computing coordinates of bounding regions encompassing respective traces. 

Claim 17 (original): The method of claim 16, wherein the computed bounding region 
coordinates are two-dimensional coordinates of areas in the acquired images. 

Claim 18 (original): The method of claim 16, wherein the computed bounding region 
coordinates are three-dimensional coordinates of volumes in the interactive space. 

Claim 19 (original): The method of claim 14, wherein the spatiotemporal input data 
structure is processed to interpret the identified input target traces. 

Claim 20 (original): The method of claim 19, further comprising comparing an identified 
trace to a predefined representation of an input gesture corresponding to a respective input 
instruction. 
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Claim 21 (original): The method of claim 20, wherein processing the spatiotemporal 
input data structure comprises translating the trace into a predefined alphanumeric character. 

Claim 22 (original): The method of claim 19, further comprising comparing an identified 
trace to a location in the interactive space corresponding to a virtual interactive object. 

Claim 23 (original): The method of claim 22, wherein the virtual interactive object 
corresponds to a virtual machine instruction input. 

Claim 24 (original): The method of claim 23, wherein the virtual machine instruction 
input is predefined. 

Claim 25 (original): The method of claim 23, further comprising constructing the virtual 
machine instruction input in response to processing of at least one identified input target trace. 

Claim 26 (original): The method of claim 23, wherein the virtual machine instruction 
input corresponds to a respective mode of interpreting traces. 

Claim 27 (original): The method of claim 1, wherein executing the identified input 
instruction comprises displaying an image in accordance with the identified input instruction. 

Claim 28 (previously presented): The method of claim 27, wherein the displaying 
comprises displaying a combination of image data generated based on the acquired images and 
machine-generated virtual image data. 

Claim 29 (original): The method of claim 27, further comprising displaying a sequence 
of images at the display location showing a virtual object being manipulated in accordance with 
one or more identified input instructions. 



Claim 30 (previously presented): A method of interfacing with a machine, comprising: 
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acquiring sets of contemporaneous images of an interactive space from multiple 
respective fields of view; 

detecting an input target in the acquired images; 

computing coordinates of the input target detected in the acquired images; 
constructing a spatiotemporal input data structure linking input target coordinates 
computed from contemporaneous images to respective reference times; 

processing the spatiotemporal input data structure to identify an input instruction; and 
executing the identified input instruction on the machine; and 

interpolating between fields of view to generate a synthetic view of the interactive space. 

Claim 31 (previously presented): A system for interfacing with a machine, comprising: 
multiple imaging devices configured to contemporaneously acquire, at each of multiple 
capture times, a respective image from each of multiple fields of view defining an interactive 
space to create a respective set of contemporaneous images; and 

a processing module configured to detect an input target in the acquired images, compute 
two-dimensional coordinates of the input target detected in the acquired images, construct a 
spatiotemporal input data structure linking each of the capture times to the computed two- 
dimensional coordinates of the input target in each of the contemporaneously acquired images in 
a respective one of the sets of contemporaneous images, process the spatiotemporal input data 
structure to identify an input instruction, and execute the identified input instruction on the 
machine. 

Claim 32 (previously presented): A machine-readable medium storing machine-readable 
instructions for causing a machine to: 

at each of multiple capture times, contemporaneously acquire a respective image from 
each of multiple fields of view defining an interactive space to create a respective set of 
contemporaneous images; 

detect an input target in the acquired images; 

compute two-dimensional coordinates of the input target detected in the acquired images; 
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construct a spatiotemporal input data structure linking each of the capture times to the 
computed two-dimensional coordinates of the input target in each of the contemporaneously 
acquired images in a respective one of the sets of contemporaneous images; 

process the spatiotemporal input data structure to identify an input instruction; and 

execute the identified input instruction on the machine. 

Claim 33 (previously presented): A method of interfacing with a machine, comprising: 
displaying an image at a display location disposed between a viewing space and an 

interactive space, wherein the displayed image is viewable from a perspective in the viewing 

space; 

acquiring images of the interactive space from a field of view directed toward the 
interactive space along an optical axis intersecting a central area of the display location; 
detecting an input target in the acquired images; 

computing coordinates of the input target detected in the acquired images; 
identifying an input instruction based on the computed input coordinates; and 
executing the identified input instruction on the machine. 

Claim 34 (original): The method of claim 33, wherein the display location corresponds 
to a display area of a portable electronic device. 

Claim 35 (original): The method of claim 33, wherein the display location corresponds 
to a display area embedded in a desktop surface. 

Claim 36 (original): The method of claim 33, wherein displaying the image comprises 
projecting the image onto a surface. 

Claim 37 (original): The method of claim 33, wherein acquiring images comprises 
acquiring images of the interactive space from at least one field of view disposed between the 
display location and the interactive space. 
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Claim 38 (canceled) 

Claim 39 (original): The method of claim 33, wherein acquiring images comprises 
acquiring images of the interactive space from multiple fields of view. 

Claim 40 (original): The method of claim 39, further comprising interpolating between 
fields of view to display an image at the display location corresponding to a synthetic view of the 
interactive space. 

Claim 41 (original): The method of claim 39, further comprising computing calibration 
parameters for the multiple fields of view. 

Claim 42 (original): The method of claim 41, wherein computing coordinates of the 
detected input target comprises computing three-dimensional coordinates of the input target in 
the interactive space based on the computed calibration parameters. 

Claim 43 (original): The method of claim 33 , wherein detecting the input target in the 
acquired images comprises comparing values of pixels in the acquired images to at least one 
threshold pixel value. 

Claim 44 (original): The method of claim 43, wherein computing coordinates of the 
input target comprises computing coordinates of centroids of respective groups of pixels in the 
acquired images with values greater than the threshold. 

Claim 45 (original): The method of claim 43, wherein detecting the input target in the 
acquired images comprises segmenting foreground pixels from background pixels in the 
acquired images. 
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Claim 46 (original): The method of claim 33, wherein computing coordinates of the 
detected input target comprises computing two-dimensional coordinates of the input target 
detected in the acquired images. 

Claim 47 (original): The method of claim 33, wherein identifying an input instruction 
comprises identifying traces of the input target in the interactive space. 

Claim 48 (original): The method of claim 47, wherein identifying traces comprises 
detecting state change events and segmenting traces based on detected state change events. 

Claim 49 (original): The method of claim 47, wherein identifying traces comprises 
computing coordinates of bounding regions encompassing respective traces. 

Claim 50 (original): The method of claim 49, wherein the computed bounding region 
coordinates are two-dimensional coordinates of areas in the acquired images. 

Claim 51 (original): The method of claim 49, wherein the computed bounding region 
coordinates are three-dimensional coordinates of volumes in the interactive space. 

Claim 52 (original): The method of claim 47, wherein identifying the input instruction 
comprises interpreting the identified input target traces. 

Claim 53 (original): The method of claim 52, further comprising comparing an identified 
trace to a predefined representation of an input gesture corresponding to a respective input 
instruction. 

Claim 54 (original): The method of claim 53, wherein processing the spatiotemporal 
input data structure comprises translating the trace into a predefined alphanumeric character. 
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Claim 55 (original): The method of claim 52, further comprising comparing an identified 
trace to a location in the interactive space corresponding to a virtual interactive object. 

Claim 56 (original): The method of claim 55, wherein the virtual interactive object 
corresponds to a virtual machine instruction input. 

Claim 57 (original): The method of claim 56, wherein the virtual machine instruction 
input is predefined. 

Claim 58 (original): The method of claim 56, further comprising constructing the virtual 
machine instruction input in response to processing of at least one identified input target trace. 

Claim 59 (original): The method of claim 56, wherein the virtual machine instruction 
input corresponds to a respective mode of interpreting traces. 

Claim 60 (original): The method of claim 33, wherein executing the identified input 
instruction comprises displaying an image at the display location in accordance with the 
identified input instruction. 

Claim 61 (original): The method of claim 60, wherein the displayed image comprises a 
combination of image data generated based on the acquired images and machine-generated 
virtual image data. 

Claim 62 (original): The method of claim 60, further comprising displaying a sequence 
of images at the display location showing a virtual object being manipulated in accordance with 
one or more identified input instructions. 



Claim 63 (previously presented): A system of interfacing with a machine, comprising: 
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a display configured to present an image at a display location disposed between a viewing 
space and an interactive space, wherein the displayed image is viewable from a perspective in the 
viewing space; 

at least one imaging device configured to acquire images of the interactive space from a 
field of view directed toward the interactive space along an optical axis intersecting a central 
area of the display location; and 

a processing system configured to detect an input target in the acquired images, compute 
coordinates of the input target detected in the acquired images, identify an input instruction 
based on the computed input coordinates, and execute the identified input instruction on the 
machine. 

Claim 64 (previously presented): A machine-readable medium storing machine-readable 
instructions for causing a machine to: 

display an image at a display location disposed between a viewing space and an 
interactive space, wherein the displayed image is viewable from a perspective in the viewing 
space; 

acquire images of the interactive space from a field of view directed toward the 
interactive space along an optical axis intersecting a central area of the display location; 
detect an input target in the acquired images; 

compute coordinates of the input target detected in the acquired images; 
identify an input instruction based on the computed input coordinates; and 
execute the identified input instruction on the machine. 

Claim 65 (previously presented): The method of claim 1, further comprising 
interpolating between ones of the images contemporaneously acquired from ones of the fields of 
view to generate a synthetic view of the interactive space. 



